stability: close validated audit findings remediation - #34

Context

This PR completes a full remediation pass for validated repository findings from two independent deep audits, with emphasis on reliability, cross-platform behavior, test signal quality, and maintainability.

The work spans core Bazel/Starlark rules, the Go companion module, integration harness behavior, CI/security workflows, and user/maintainer docs.

What Changed

Uploader hardening and maintainability

• Refactored the uploader implementation by extracting large embedded script templates from tools/core/test_optimization_uploader.bzl into dedicated template modules:
  • tools/core/uploader_bash_template.bzl
  • tools/core/uploader_powershell_template.bzl
  • tools/core/uploader_batch_template.bzl
• Kept rule behavior consistent while making templates reviewable and easier to test independently.
• Added explicit default public visibility via the dd_payload_uploader macro wrapper.
• Hardened PowerShell upload behavior for CI reliability:
  • explicit boolean-return semantics for upload helpers,
  • suppression of incidental pipeline outputs from .NET Add() calls,
  • robust stream-result handling,
  • fixed ByteArrayContent constructor usage for gzip payload bytes.

Sync/multi-sync and bootstrap robustness

• Added HTTP policy attribute parity to multi-service sync extension (http_* timeout/retry knobs) so multi-service users can configure the same transport behavior as single-service users.
• Refactored sync env collection logic for readability/maintainability (including DD_GIT override handling extraction).
• Fixed deep-copy behavior in payload cloning paths to prevent nested attribute aliasing.
• Strengthened go bootstrap path validation (non-empty, relative path constraints, traversal checks, and explicit MODULE.bazel existence validation at target path).

Go companion rule behavior

• Hardened importpath inference in modules/go/topt_go_infer.bzl with type checks.
• Extended inference aspect traversal from embed-only to embed + deps to reduce missed inference scenarios.
• Added label-shape validation in dd_topt_go_test module selection inputs.
• Normalized rundir handling to deterministic package-relative execution semantics.

Test coverage and integration depth

• Expanded Starlark unit coverage across core helpers and edge cases, including:
  • manifest parsing and BOM/whitespace helpers,
  • CI provider env mapping and ref normalization,
  • module label mapping edge cases,
  • payload deep-clone behavior,
  • common util behavior (sanitize_label_fragment, dedup_keys, logging helpers).
• Expanded Go companion tests for selection utility paths, failure cases, and aspect traversal.
• Expanded Python tests for:
  • schema validation internals,
  • schema sync fallback behavior,
  • module version checker failure paths.
• Strengthened integration harness + mock server scenarios:
  • sustained/partial HTTP failure paths,
  • retry reset semantics,
  • coverage payload validation stricter fixtures,
  • clearer Windows wrapper diagnostics,
  • more deterministic gzip/failure-path assertions across platforms.

CI/security workflow improvements

• Hardened coverage signal in CI:
  • replaced non-actionable path with direct Python trace-based probe for tools,
  • fail-fast on zero executable lines,
  • raised meaningful minimum threshold.
• Added explicit minimal workflow permissions where missing (contents: read).
• Expanded CodeQL coverage matrix to include actions in addition to python.
• Corrected Chainguard STS repository mapping mismatch.
• Documented Dependabot scope rationale.

Documentation and examples alignment

• Updated maintainer/user docs to align with actual repository command patterns and constraints.
• Clarified uploader and configuration behavior (DD_TEST_OPTIMIZATION_FILTER_PREFIX, HTTP -1 semantics, multi-service uploader usage).
• Aligned and clarified example module guidance (including local override workflow notes).
• Added/updated troubleshooting and contributor guidance around parser diagnostics and timeout metadata hygiene.
• Removed the temporary internal remediation register doc to keep repository docs focused on durable product/maintainer guidance.

Compatibility / Behavior Notes

• No breaking rename of stable generated labels.
• Existing single-service and multi-service flows remain supported; this PR primarily improves parity, correctness, and diagnostics.
• Windows uploader path is significantly more robust under CI/runtime edge conditions.

Validation Matrix

• ./bazelw test //tools/...
• ./bazelw build //examples/...
• cd modules/go && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../..
• tools/tests/integration/run_mock_server_tests.sh
• python3 tools/dev/check_module_versions.py
• ./bazelw test //tools/... --spawn_strategy=sandboxed --strategy=TestRunner=sandboxed --sandbox_default_allow_network=false --modify_execution_info=TestRunner=+block-network --test_env=TZ=UTC --test_env=LANG=C --test_env=LC_ALL=C --enable_runfiles
• cd modules/go && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../.. --spawn_strategy=sandboxed --strategy=TestRunner=sandboxed --sandbox_default_allow_network=false --modify_execution_info=TestRunner=+block-network --test_env=TZ=UTC --test_env=LANG=C --test_env=LC_ALL=C --enable_runfiles

Risk / Rollback

• Primary risk area is cross-platform uploader runtime logic (especially PowerShell transport path), mitigated by new targeted unit/integration coverage.
• Rollback strategy is straightforward: revert this PR branch if regression signals appear in uploader or integration lanes.